Talk by Ross Gayler for the Redwood Center for Theoretical Neuroscience at UC Berkeley.
ABSTRACT. This talk is about computing with discrete compositional data structures in analog computers. A core issue for both computer science and cognitive neuroscience is the degree of match between a class of computer designs and a class of computations. In cognitive science, it is manifested in the apparent mismatch between the neural network hardware of the brain (essentially, a massively parallel analog computer) and the computational requirements of higher cognition (statistical constraint processing with compositional discrete data structures to implement facilities such as language and analogical reasoning). Historically, analog computers have been considered ill-suited for implementing computational processes on discrete compositional data structures.
Neural networks can be construed as analog computers -- a class of computer design with a long history, but now relatively unknown. Historically, analog computation had advantages over digital computation in speed and parallelism. Computational problems were cast as dynamical systems and modelled by differential equations, which was relatively straightforward for models of physical problems such as flight dynamics. However, it was far less clear how to translate computations on discrete compositional data structures such as trees and graphs into dynamical systems. This is especially true for problems where the data structures evolve over time, implying the need to rewire the analog computer on the fly. This is particularly relevant to cognitive science because new concepts and relations can be created on the fly, and under the standard conception of neural networks this implies that neurons and connections would be created impossibly rapidly.
In this talk I describe Vector Symbolic Architectures, a family of mathematical techniques for analog computation in hyperdimensional vector spaces that map naturally onto neural network implementations. VSAs naturally support computation on discrete compositional data structures and a form of virtualisation that breaks the nexus between the items to be represented and the hardware that supports the representation. This means that computations on evolving data structures do not require physical rewiring of the implementing hardware. I illustrate this approach with a VSA system that finds isomorphisms between graphs and where different problems to be solved are represented by different initial states of the fixed hardware rather than by rewiring the hardware. Graph isomorphism is at the heart of the standard model of analogical reasoning and motivates this example, although that aspect is not explored in this talk.
BIO. Ross Gayler has PhD in psychology from University of Queensland, with a long and far-reaching interest in the mysteries of cognitive computing. His day job is as a statistician in the finance industry.
